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Improving Schools Study 
Executive Summary 

The new ABCs Accountability Model places new demands on 
individual schools to improve student performance. As the ABCs 
Accountability Model is implemented, the Evaluation Section will be 
looking at high performing schools and schools that show strong 
improvement over time, studying what they are doing to enhance 
student learning, and sharing these findings with other schools 
throughout the state. As a beginning study, eleven high poverty 
schools (above state average on free-reduced price lunch [FRL]) 
demonstrating success in moving students up the four Achievement 
Levels on the End-of-Grade (EOG) Tests were studied. 

The Index was the measure of improvement used in this study. 
Increases on the Index indicate that students are moving up the four 
Achievement Levels, although that number alone does not indicate 
which levels are most impacted. See Appendix C for explanation. 

Above state-average FRL schools were selected where at least one 
cohort had demonstrated exemplary on both reading and mathematics 
over one year or exemplary growth in one subject area over a two- 
year period. The study did not attempt to find the best schools in the 
state or the highest growth schools. Evaluators attempted to find 
“typical” schools that were demonstrating fairly consistent growth 
and that might provide insight for other such schools. 

Selected schools were visited by two-person teams for one day. 
Interviews were held with the principd and a group of teachers, 
chosen by the principal, who had contributed to the growth of 
students. 

The following characteristics were found in at least several schools, 
not necessarily every one. Categories represent major commonalities 
found among the schools. Some schools may have exhibited more 
of one trait than another, but - taken together - these characteristics 
seemed to be more typical than not. 

• High expectations are universal . Staff in these schools really 
believed that all children can learn well and took it as their 
responsibility to see that they did. Expectations were 
communicated in a number of ways. There were no excuses for 
low performance. 

• Care and respect are clearly evident . School staff care about their 
students; respect is highly valued. Staff were accessible. 

• The environment is orderly and safe . This often cited “effective 
schools” correlate was clearly evident in these schools. Discipline 
problems were infrequently mentioned; and, when probed, 
teachers indicated that it was not a serious problem. 



• Reform is teacher-driven . Reform happens in the classroom. 
Results of individual teachers were evident. The changes in one 
school had been initiated by teachers even lacking the 
administrative support evident in other schools. 

• The principal leads, supports, and promotes teamwork . Most - 
but not all - schools reflected a strong school-based management 
philosophy. Most principals had a clear vision of how the school 
should operate. Teachers had input into how things were done, 
and principals tended to be actively engaged with students. 

• Assessment is important and is used . All forms of assessment 
played an important role in these schools. Staff studied the EOG 
Tests results, used testlets and other diagnostic strategies, linked 
their classroom work to the test Achievement Levels, used test 
item banks, and used commercially and locally developed reading 
inventories. School staff paid attention to assessment results as 
feedback regarding student performance and instructional needs. 

• Focus on the Standard Course of Study ('SCSI . Almost every 
school reported adhering to the SCS. Teachers made sure they 
understood the SCS goals in core subject areas. Staff 
development for some included alignment with the SCS, 
“unpacking” the SCS goals for better understanding, and the like. 

• Instruction: Both didactic and hands-on . No single form of 
instruction emerged from these case studies. Many staff had 
moved to more manipulatives and hands-on instruction and more 
teaching in context/whole language. As a consequence they 
reported that students had much better understanding of concepts 
than previously. However, most schools still used phonics and 
skills instruction as part of their instruction. Many classrooms 
were very traditionally structured. 

• Provide extra help and early intervention . Schools found ways to 
provide assistance at the first signs of trouble, as well as offering 
tutoring and other sources of extra instruction. 

• Be willing to experiment . Staff in these schools were typically 
willing to work together to try new approaches. They were aware 
of research, but used it in combination with their experience and 
what they determined was best for their students. If their first 
attempt was unsuccessful, they were willing to try another 
strategy. 



Do Whatever 
It Takes 



The attitude and efforts of these schools can best be summarized as 
“doing whatever it takes.” Many staff give 100+ percent effort and 
did not give up on students. 



Next Steps 



The Evaluation Section will continue to study ABCs Accountability 
Model results to identify high growth and high performing schools. 
Schools will be studied to help inform the schools improvement 
process and to improve student learning. 
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Improving Schools Study: 

Another Way to Look at Raising Student 

Achievement 



Introduction 

Purpose of Study 

The new School-Based Management and Improvement Program - the State Board 
of Education’s ABCs Accountability Model - places even greater demands on individual 
schools to improve student performance continuously on state End-of-Grade (EOG) 

Tests, grades 3-8. While these tests are not the only way to measure successful 
learning, they are one indicator of student learning and a key ingredient for school 
accountability for elementary and middle schools. The Evaluation Section of the 
Department of Public Instruction was interested in using the state EOG Tests to identify 
schools that were already improving student performance on a steady basis, to study 
those schools, and to share the kinds of strategies used by these schools with other 
schools in the state. 

The ABCs Accountability Model 

In the ABCs Accountability Model, the A stands for Accountability, the B stands 
for Basics (reading and mathematics), and the C stands for local control. The ABCs 
Accountability Model in grades K-8 is based on cohort growth from year-to-year on the 
states EOG tests. The model is school-based; that is, the school is the unit evaluated for 
accountability purposes. ABCs Tools, software to run the formula calculating the 
expected and actual growth for each school, was developed and finalized in 1996. The 
amount of growth for each school is established based on a regression formula that 
considers the average state growth the year the tests were normed, as well as a" 
regression factor. The actual growth among cohorts for the year is measured against the 
predicted (or expected) growth to determine if the school met expected growth. Schools 
that exceed expected growth and make exemplary growth (e.g., 110% of state average in 
the regression formula) will earn bonus money to be used as determined by teachers in 
the school. Schools that achieve 100% of expected growth earn special recognition and 
schools that do not achieve expected growth receive technical assistance. There are also 
other categories of recognition and sanctions. 

Initial Stages in the Study 

This Improving Schools Study began in late 1995, even before the ABCs Tools 
software for calculating cohort growth on the EOGs had been finalized. By using the 
initial formula and looking at grade-by-grade cohort growth, schools were identified 
that met the following criteria; (1) they had one or more grade-level cohort groups that 
had exceeded their expected growth on EOG Tests by 110% for two years in a row in one 
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subject area (e.g., reading or math)or (2) they had made 110% growth for one year on 
both reading and math. Due to the data base available at that time, grade 6 was omitted 
from these analyses. 

The first strategies for examining these data collectively included traditional 
statistical procedures such as factor analysis and discriminant function analysis. A 
factor analysis was conducted using a number of school variables that were available in 
various state data bases (e.g., schools size and space variables, class size, percent 
free/reduced price lunch, percent minority, teacher characteristics). These procedures 
showed that there were clear factors related to cohort growth performance, the most 
heavily weighted factor (socio-economic status) being composed of socio-economic 
indicators for students (i.e., percent FRL, percent Title I, and percent minority). 

Initial findings indicated that - in general - schools that had a lower percentage of 
students eligible for free/reduced price lunch and a higher percentage of white students 
(both factors being related) had more cohort groups of students increasing achievement 
by 110%. The second most heavily weighted factor (School Space) dealt mostly with 
factors related to space and size (i.e., average age of building, total square feet, average 
square feet per student, number of buildings). 

While these initial findings were not surprising given research on the 
relationship between achievement and socio-economic status (SES), it was not 
especially helpful in identifying schools and strategies to improve student achievement 
over time - especially given the relatively high number of poor and high-minority 
schools in North Carolina. In addition, most of these factors are not variables that the 
school can manipulate to raise achievement. Thus, the search for a strategy to identify 
schools successful in spite of these SES and space variabies was continued. 



An Agency- Wide Study Team 

Improving student performance is the primary concern of the Department of 
Public Instruction (DPI). Most of the DPI Divisions and Sections have responsibilities 
in this area. Thus, an agency cross-functional team was formed for the “Improving 
Schools Study” as the search to identify evaluation strategies continued. Members of 
that team (see Appendix A) came from the Divisions of Accountability Services, School 
Improvement, Instructional Services and Information Management, and the Office of 
Education Reform. Team members met periodically to provide input and guide the 
direction of the study, to assist the Evaluation Section staff in making decisions about 
how to proceed, to review data and interview protocols, and to participate in the visits to 
the schools to conduct the case studies. 



Focusing the Study 

Given the limited usefulness of the earlier analyses, the Evaluation staff and ’ 
cross-functional team focused on what was the most important issue in this study. We 
decided that finding ways to heip iower performing, high poverty schoois to improve 
student achievement was the most useful goal for this study. Thus, the focus became 



schools that were above the state average on percentage of students eligible for 
free/reduced price lunch that also showed evidence of raising student achievement. The 
FRL percentage was used as a proxy for lower SES (higher poverty). 

At about this stage in the study, the usefulness of the “Index” was discovered. 

The Index is a measure that was used to compare different cohort groups for two years on 
state high school End-of-Course Tests for 10 ABCs Accountability Model pilot sites 
during the 1995-96 school year. The Index turned out to be a succinct and helpful 
measure in determining cohort growth on the End-of-Grade tests as well, and also 
proved to be a useful program evaluation tool for schools. This measure was used to 
select schools for the Improving Schools case study schools . 



The Index 

What is the Index? 

Technically, the Index is the sum of the weighted number of students in the four 
achievement levels on the EOG tests. The EOG tests have four Achievement Levels for 
each grade: Level IV (student is performing well above grade level); Level III (student 
is performing solidly at grade level); Level II (student is performing inconsistently at 
grade level, mastery is not yet attained); and Level I (student is performing well below 
grade level). It assigns a weight of “3” to Achievement Level IV, a weight of “2” to 
Achievement Level III, a weight of “1” to Achievement Level II, and a weight of “0” to 
Achievement Level I. Thus, schools get no credit for students in Achievement Level I. 

The weights for each level are multiplied by the percentage of students in that level, and 
the sum is divided by 3 (see Appendix B). 

The Index ranges from 0-100. In an example scenario, if a grade level or school 
has a score of “0”, that would mean that all students are in Achievement Level I. A 
score of 100 on the Index would mean that all students are in Achievement Level IV. A 
score of 50 would suggest that about half the students are in Levels I and II. and half are 
in Levels III and IV. However, the actual configuration of student scores could vary 
widely with any given Index score. Appendix C illustrates the Index continuum from 0 to 
100 and Index scores for several sample student distributions across Achievement 
Levels. The primary purpose of the Index is to show the general location of 
students on Achievement Levels and to show growth in the number of students 
from one Achievement Levei to another. Thus, when the Index score goes up, 
more students have moved to higher Achievement Levels. The Index does not indicate 
which Achievement Levels are involved, but does indicate if, overall, students are 
moving to higher Achievement Levels (or, conversely, to lower Achievement Levels). If 
the Index does not change from one year to the next, then - while Developmental Scale 
Scores on the EOG Tests may have increased - the distribution of students across. 
Achievement Levels had essentially remained unchanged. 

Relationship of the Index to Other Measures 

The Index is obviously related to, but does not exactly parallel, growth in the 
developmental scale scores (DSS) on the state’s End-of-Grade (EOG) tests, grades 3-8. 
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While the DSS measures continuous growth, the Index only measures growth when 
movement is made from one Achievement Level to the next. The DSS may increase from 
one year to the next without a corresponding increase in the Index (i.e., higher 
percentage of students in higher Achievement Levels). On the other hand, if the Index 
increases, the DSS likely has increased. Correlation with the expected growth index for 
an entire school used in the ABCs Accountability Model is moderate (around .5 or .6 when 
we look at whether the expected growth is “made” or “not made”), but much higher for 
individual grade-level cohort growth expectations (.8 to .9). Thus, the Index may show 
movement of students up the Achievement Levels when the school-level ABCs . 
Accountability Model will not show growth. Therefore, the Index provides additional 
information about student and school progress and is a quick reference to determine if 
the school is being successful in improving achievement for individual students and to 
what extent students are moving up the Achievement Levels. It is a more diagnostic 
measure than the ABCs Accountability Model of growth. 

Implication of the Index 

The importance of the Index lies in its ability to provide a quick look at the extent 
to which a school is moving increasing numbers of students from lower into higher 
Achievement Levels on the EOG tests. While increases are expected in DSS from one year 
to the next, it is not readily apparent the extent to which these gains are being made by 
lower performing students moving into higher achievement levels. Certainly the ABCs 
exemplary growth is one measure of “more-than-expected” growth. The addition of the 
Index as an analysis and evaluation tool provides a clearer picture of the student 
distribution across Achievement Levels for a given cohort group. Increases in the Index 
clearly show that more students are performing at higher Achievement Levels. And the 
higher the Index score, the greater the percentage of students in the upper Achievement 
Levels. We will never reach our high standards for all students unless schools - 
especially poor and lower performing ones - demonstrate consistent Index increases 
each year. 



Selecting Schools for the Study: Growth and the Index 

“Improvement” in achievement was defined by “growth.” Growth is defined as 
the amount of increase expected on EOG test scores using the formula developed for the 
state ABCs Accountability Model. Because the new ABCs Accountability Model uses 110% 
“growth” as the criterion for exceeding expected growth, this criterion was used as 
“above expected” growth for this study. First the lower SES schools in our original data 
base were reviewed to select the schools that had one cohort group that (a) had made 
110% growth over two years for one subject o r (b) had made 110% growth for one 
year in both reading and mathematics.- Appendix D describes the eight definitions or 
criteria (A-H) of “above expected growth” used to identify improving schools. Each 
time a school met one of these eight criteria, a “YES” was indicated. With the 
calculations by grade level that had been completed earlier, schools with grades K-6 had 
a possible maximum of four “YES’s,” as did schools with grades 6-8. Schools with 
grades K-8 had a maximum of eight possible “YES’s.” 
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Where possible schools were selected that had met two (out of four) of these 
criteria. However, very few schools met two criteria, especially middle schools. Thus, 
one “YES” was used to select some schools, focusing on schools that would give more 
geographic and ethnic distribution. Finally, a couple of schools were included in the 
study that were nominated by cross-functional team members, based on their work with 
schools undergoing reform. One school, for example, had been working with the 
“Effective School” correlates for several years. Once a core group of schools was 
identified that had one or more “YES’s,” the Index was used to further study cohort 
growth for those schools across aM grade levels. Ultimately eleven schools were selected 
that appeared to have reasonably consistent growth in Index scores across grade levels 
for more than one year and/or cohort group. The last column in Appendix E shows the 
number of “YES’s” each school included in the study actually attained and the maximum 
number that could have been attained based on the number of grade levels in the school. 

It is important to note that this study was not trying to identify the absolute best 
schools in the state, but rather typical high poverty schools that were showing good and 
steady growth across Achievement Levels. This is a preliminary study for further work 
in the future. Index scores were calculated for the total student population in each grade 
level, as well as for the predominant ethnic groups in the school. To check each 
selection. Achievement Level distribution was disaggregated by ethnic group and, indeed, 
more students tended to move to higher Achievement Levels for all subgroups for these 
schools. It is also important to note that the Index did not increase for every cohort 
group or for all ethnic groups every year. Indeed, some decreases were found for some 
years and/or ethnic groups. Our interest was in schoois that had a preponderance of 
increases and that tended to exceed index growth for the state as a whoie. 



What the Schools Looked Like 

Eleven schools were selected for further case study. Time was not spent trying to 
find the highest Index growth in the state in selecting these schools. Rather, the focus 
was on identifying typical higher poverty improving schools. Essential school 
characteristics are found in Appendix E. With one exception, all schools exceeded the 
state average (about 50%) for the percent of students eligible for ‘Iree/reduced price 
lunch.” The actual percentage ranged from 52.9% to 92.3%. Many of these schools also 
had a high percentage of minority students. While the state student population average is 
around 30% Black and 1 .5% Native American, these schools (with one exception) ranged 
from 44.6% to 98% minority. The one exception was a mountain school with only 2.2% 
minority, a typical percent for that geographic region of the state. The most prominent 
minority group was black students; however, two of the schools had significant Native 
American populations. 

The schools were mostly rural and suburban schools, with one school located in a 
large city. They were located in eastern, central, and western parts of the state. Seven 
were elementary schools consisting of grades PK, K-5, or K-6 (one of grades 3-5); two 
were middle schools (grades 6-8); and two were both elementary and middle schools 
(grades PK-8). Schools ranged in size from 178 to 805 students (both elementary 
schools). Grade configuration was not predictive of school size. However, it is 



important to note that several smaller schools had only one or two teachers per grade; 
thus, teacher effects were clearly being measured. 



Improving Schools’ ABCs Accountability Model and Index Growth 

Appendix F shows the status of schools using the ABCs growth formula. However, 
it must be emphasized that, because we used the state data base, these calculations were 
done without matching students from one year to the next and without eliminating 
students who were not in attendance for at least half the year. Also, the growth 
expectations include only reading and mathematics portions of the EOG tests, not writing. 
Nevertheless, these data show that schools that evidenced reasonably consistent Index 
growth for individual cohort groups had variable success as a whole school on the ABCs 
growth model. Six schools “met” expected growth for each year in this study; four met 
expected growth for two out of three years; and one met expected growth for .only one out 
of three years. This last school was one school selected later in the process to fill out a 
particular ethnic representation and was not among the higher Index growth schools. 

Results were more variable for “exemplary” growth (i.e., 110% growth). Only 
two schools showed exemplary growth for all three years. Four had exemplary growth 
for two out of three years. Two schools had exemplary growth for only one year, while 
three schools never met the exemplary growth standard. 

Appendix G shows the amount of Index growth for the 1 1 schools and the state as a 
whole by total student body, white students, black students, and Native American 
students. Index growth (or loss) is shown for both reading and mathematics. Across both 
subjects, there are a total of 108 cohort growth comparisons between the 11 schools and 
the state' (30 each for total, white, and black cohorts; and 18 for Native American 
cohorts). Of these 108 comparisons, the 11 schools exceeded the state average by more 
than one Index point on 68 (almost two-thirds of) comparisons. They were within one 
Index point on 14 comparisons and fell behind the state by more than one Index point on 
26 comparisons. Mathematics seemed to have more lower growth comparisons than 
reading across all ethnic groups, but especially for black and Native American students. 
However, this may reflect the fact that many of these schools had an emphasis on reading. 
Also, negative comparisons were more plentiful in the 1995 and 1996 years for sixth 
and seventh grades. Thus, using the ABCs growth formula to look at individual cohort 
growth did identify schools that had generally higher Index growth than the state average 
for the majority of cohort groups. 



Case Study Procedures 

Two-person teams were formed for each of the eleven schools. An interview 
protocol was developed, and school systems and schools were contacted. The 
superintendent was contacted first to inform him/her that one or two schools in that 
particular district had been identified as consistently improving student achievement 
over time and to explain the purpose of the study. Given approval by the superintendent, 
the team then contacted the principal. Interest in participating was received from both 
the superintendent and the school principal, and visits were scheduled. Each visit ranged 
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from a half to a whole day. The principal was interviewed alone. Also, a group of 3-5 
teachers was interviewed in a focus group format. The principal was asked to select 
these teachers from among those teachers the principal thought were contributing most 
to their improving scores. At one K-8 school, three separate groups of teachers were 
interviewed by grade-level groupings (e.g., primary, elementary, middle). 

When the interviewers met with the principal, the results of the Index growth 
for each grade cohort across years was shared, along with the disaggregated data for each 
Achievement Level. Ways to use the Index to evaluate the school program, weaknesses in 
subject areas, or grade-level problems were shared. 

The interviews per se were designed to pose only general questions, such as 
“Why do you think your school has been successful in improving student achievement?” 
Probes were used to follow-up unclear comments or to ask about specific areas not 
addressed by open-ended prompts. The intent was not to direct the conversation but to 
let each school “tell its own story.” 

One member of each team wrote a school report on returning from the site visit. 
The team members met as a group after all visits had been completed to discuss their 
findings for the schools visited. The commonalties across schools, as well as any unique 
aspects, were noted. This report is a synthesis of those findings. 

It is important to note that the findings are based on brief, one-day observations 
of the interviewers and the self-report comments of the principals and teachers 
involved. There was no opportunity to do intense validation of the comments. In a couple 
of instances, a case study team member noted that what school staff said did not seem to 
be corroborated by their brief class and/or school observations, but that was the 
exception rather than the rule. However, we did not have time to visit classrooms over 
time, and these results must be considered with the understanding that they are based on 
the limited visits. Still, the commonalties found across schools were so striking that 
they should not be dismissed. These schools reported some very similar philosophies, 
points of view, and strategies. 

What Did We Find? 

Index Growth Highest at Elementary Grades 

Appendix G shows the aggregated amount of Index growth by grade level for all 11 
schools compared to the average state Index growth. Index gains are also disaggregated 
for black, white, and Native Americans students. The greatest amoynt of growth is 
evident at the elementary grades (3-5). In fact, grade 4 had the overall highest amount 
of growth, followed by grade 5, across three different cohort groups especially for black 
and white students. Native Americans followed this pattern in grade 4, but results were 
more varied for grade 5. These gains were not as true for the state as a whole, although 
grades 4 and 5 tended to have higher statewide gains in mathematics than other cohort 
groups and subjects. Where negative Index changes (declines) were found, they tend to 
be attributed primarily to one school. 




Several sizable gains at some middle grades were found for selected cohort groups 
compared to statewide data. For example, black students at the 8'^ grade in both 1 995 
and 1996 made higher gains than white students in both reading and mathematics. 
However, when larger losses on the Index were found, they were typically found for 
black or Native American students. Gains for Native American students were the least 
consistent across grade levels. There were some declines in the Index scores at several 
middle grades (6-8) levels for certain cohort groups. This finding is consistent with 
other statewide data indicating less growth at the middle grades on reading and 
mathematics EOG tests. 

Characteristics Found in These Schools 

The characteristics presented here were found in at least several of the schools, 
not necessarily every one. These categories represent major commonalties found among 
the schools. Different schools might have illustrated more of one characteristic than 
another, and different strategies might have been used across schools to illustrate the 
same characteristic or concept. On the whole, however, these characteristics seemed to 
be more typical than not. Finally, while the observations are sorted into discrete 
categories, it should be noted that the characteristics noted cut across these categories. 
For example, the placing of student work on the wall is addressed under safe and orderly 
environment, but it also relates to motivation and citing high-quality work. 

High Expectations are Universal. This finding may not be surprising 
given the current emphasis on high standards and expectations for all students. 

However, the phrase “All children can learn.” is easily said but less easily 
implemented, and often not truly believed when actual practices of schools are examined. 
However, in these case studies we found schools whose staffs really believed that their 
students could learn and learn well. They also believed that it was the staff’s 
responsibility to ensure that they did so. The principals and teachers had very clear 
expectations of their students, from clearly posted classroom rules and class work to 
school-wide strategies. 

High expectations were communicated in numerous ways. For example, one 
school with a high percentage of low income students had banners of various colleges 
posted in the school lobby; the message was that students here were college-bound. 
Mottoes and slogans in several schools conveyed their expectations, such as the following: 
“We care. We cooperate. We continue to learn;” and “Dream. Believe. Strive. Succeed.” 
Uniforms worn in one school were designed to communicate that students are coming to 
school to work, just as their parents put on “work clothes.” Another principal believes 
that “every child needs to develop a work ethic. No child is allowed to sit and do 
nothing.” 

In general, teachers in these schools do not let students do easy work to feel good 
about themselves. They want their students to feel good about themselves because they 
were competent and capable learners. Teachers provide consistency and predictability 
for students. They “stick to their guns with students” and do not back down. 

No excuses were allowed for low performance. This came up in almost every 
interview. Teachers and principals noted that, while parental participation was often 
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not what they wished, the students’ poor home environments were not an excuse for poor 
performance in their schools. One school noted that there was often little structure in 
their students’ lives at home and that, therefore, the school had to provide it. Other 
schools did not even talk about family deficits or needs. They tried to enhance parent 
involvement but did not wait for it. 

Care and Respect are Clearly Evident. Teachers and principals in these 
schools care about their students; respect is highly valued. They are committed to 
students and will do “whatever it takes.” As noted above, they were also concerned 
about self-esteem of their students but not by letting students do easy work. One teacher 
whose classes showed consistent Index growth year-after-year had extremely high 
growth for one class in particular. She noted that when this class of students came to 
her, they would not try to do hard work. After a few weeks of frustration, she talked to 
them about why they would not try. They essentially told her that they had learned they 
could not do the work. She assured them that they were as capable as any of her students, 
and she would help them learn. After that conversation they began to work, with this 
class showing among the highest Index growth for one year found in this study. The 
interviewers noted that there was likely more than conversation that motivated these 
students. This teacher clearly cared about these students and implemented strategies 
that reflected that care. She was typical of many of the teachers interviewed. 

Staff in these schools were accessible. Many of them arrive early, stay late, and 
work on Saturdays. “If kids know you care, they will do anything for you.” Students 
care in return. Several interviewers noted that students in many of these schools 
seemed comfortable there. They liked their teachers and/or principal. One observer 
noted that students willingly approached one principal in a playful manner. The 
principal showed physical affection in an appropriate way. In a number of schools, 
students were interested in and took care of the school. 

Respect was also evident among the school faculty. In one school, even an 
announcement began with “Please dismiss....” rather than the more demanding 
“Dismiss....” 

Orderly and Safe Environment: This often cited characteristic of “effective 

schools” was also true of the schools in this study. High expectations were reflected in 
the physical school environment. The climate was safe and orderly; the age of the 
facility did not determine this climate. Many buildings were old, but the schools were 
clean. Many, but not all, of the schools had a lot of student work on the wall, especially 
drawing attention to high-quality work. 

Discipline was infrequently mentioned spontaneously in the interviews. When 
probed about this issue, the staff often looked surprised and commented, “It isn’t a 
problem.” Expectations about behavior and rules typically were made clear at the 
beginning of the year, and consequences were clear and quickly implemented when 
necessary. The schools were characterized by structure. Students were seldom seen 
roaming the halls or out of place in any way. A number of teachers and principals noted 
that they tried to emphasize positive consequences for appropriate behavior, as well as 
negative consequences for inappropriate behavior. 
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In several schools, every adult had equal authority with the students. All adults, 
including janitorial and cafeteria staff, were involved in establishing a positive learning 
environment and in disciplining students. These staff members knew many students by 
name and dealt with any discipline problems on the spot. In one school, for example, the 
school janitor worked with the boys who were having a problem keeping their bathroom 
clean. 



Reform is Teacher-Driven: It is trite, but true, that reform happens 

primarily in the classroom, in the interactions between teacher and student. In these 
small schools, it is easy to see the results of the work of individual teachers. Because 
some of these schools were small, the Index gains could be attributed clearly to one or 
two teachers at each grade level. These findings reinforce the understanding that the 
knowledge, skills, and care that teachers apply in the classroom will result in improved 
student learning. 

One school stood out in the extent to which the changes were initiated by teachers. 
The relationship with a relatively new principal was not close; in fact, the teachers felt 
s/he was not at the school enough to know what was going on or to provide leadership. 
However, these teachers had been the driving force behind the changes at the school 
begun a few years earlier. They indicated that the school was not doing well, and they 
knew that something had to be done. Several teachers across grade levels decided to be 
the ones to do it. They sought training and staff development in new ways to teach, and 
shared what they learned with their colleagues. Several indicated that they had totally 
changed the way they had previously taught, moving to more “hands-on” approaches in 
math and combining whole language with phonics in reading . 

The Principal Leads, Supports, and Promotes Teamwork: Most of the 

schools in this study reflected a school-based management philosophy. Styles of these 
principals varied widely. Some were truly collaborative and treated teachers as peers; 
others exercised more authority and control. Although it was apparent that some 
principals were truly more collaborative than others, teachers in most schools indicated 
that they had significant input into how things were done in their schools. Many of them 
determined how the budget was spent and had input into hiring decisions. The entire 
faculty and staff were a team. One observer commented, ‘They are on the same wave 
length.” 



The principal in these schools, for the most part, had a clear vision of how the 
school should operate and what expectations s/he had for the school. Some apologized for 
having such strong opinions about schooling or curriculum; but nevertheless, they 
conveyed their opinions to their staffs and the schools operated following these ideas. 
These administrators were constantly involved with the daily operations of their schools. 
They used terms like “I inspect what I expect,” “I use management by walking around,” 
and “I am a very hands-on principal.” 

One principal noted that, as a high school teacher, he had been the one responsible 
for what students learned and how they scored on the state End-of-Course Tests. He, 
therefore, knew that his teachers were the ones who would make the difference (or not) 

, for their students. He believed that his job was to provide them complete involvement in 



decisions about how money was spent and how instruction was delivered, and that it was 
his job to support them in any way possible. That attitude was more typical than not. 

Many principals were actively engaged with the students. They held conferences 
with students who were not doing well, contacted parents to discuss student progress, 
and treated students with love and respect. 

Assessment is Important and is Used: Assessment of all forms played an 
important role in these schools. Generally, these were child centered and data driven. 
They used data from a range of sources including data from their own diagnostic tests to 
commercial programs and the state-developed End-of-Grade (EOG) Tests. Teachers at 
these schools indicated that they took the state EOG Tests seriously. They did not seem to 
spend time arguing that these tests did not measure the “right” things. They knew the 
importance of these tests for accountability and made as much use of them as possible. In 
some cases, they may have over-interpreted test results, or used them in ways that 
stretched their reliability for individual students. Nevertheless, they paid attention to 
the results and used them to improve instruction. For example, they used the ‘Testlets” 
provided by the state to measure objectives from the NC Standard Course of Study that 
were used to develop the EOG Tests. 

Many of the schools used commercial and locally developed Instructional Reading 
Inventories and computerized test item banks in addition to state-developed diagnostic 
materials (e.g., Testlets and Linking Documents). They assessed students along the way, 
not just at the end of the year or in the period just prior to the EOG Testing. Other 
strategies were found at individual schools. For example, one teacher “graded” her 
students work according to her understanding of the Achievement Levels as they are used 
on the EOG. This practice gave her students some idea of how their work stacked up to 
grade level performance on EOG Tests. Another school had an “Assessment Room,” where 
results of EOG Tests and criterion-referenced tests were posted. One entire wall of this 
room was devoted to the management of a daily reading program and students’ placement 
in and movement through the program. 

Focus on the NC Standard Course of Study: The EOG Tests in 
reading/language arts and mathematics are based on the North Carolina Standard Course 
of Study (SCS). One finding that was observed in almost every school studied was a close 
adherence to the SCS. Teachers made sure they understood SCS goals in core subject 
areas. Some had worked with universities or other consultants to understand the goals 
and objectives more completely, especially in reading/language arts. Many of the 
schools had developed local curriculum alignment documents, pacing guides, and 
curriculum supplements. 

However, staff were very clear that the SCS was the basis for all that they taught. 
In one school, everyone - including special teachers - focused on the state SCS. For 
example, when students were studying beginning sounds in their daily reading groups, 
the physical education teacher had soccer terms with those beginning sounds and worked 
with the students to learn them. 

Instruction: Both Didactic and Hands-On: In these schools, there was 

not one single way of providing instruction. Staff sought training in how to teach the 
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curriculum objectives, leading many to move increasingly to manipulatives and hands- 
on instruction in mathematics. Almost every school indicated that it used phonics 
instruction. Many schools combined phonics with a whole-language approach, with 
schools differentially tipped in one direction or the other. However, most of these 
schools said they found that skills instruction was essential for many of their students. 
Structured approaches to reading were more typical than not. 

Almost all teachers were excited about their increased use of manipulatives in 
teaching mathematics. They found that students now seemed to understand math, not just 
go through the calculations. Many teachers were trying to find ways to apply what 
students learned to real life situations. One primary teacher said: “My students can do 
things I never would have thought possible in earlier years. My expectations have 
totally changed (i.e., increased ) for students.” 

These schools generally had lots of adult “assistants” - tutors, volunteers, extra 
part-time teachers. In several schools, every teacher in the school taught reading 
during a school-wide reading class period. Other schools developed creative schedules to 
allow for more time devoted to the core (reading and mathematics) subjects. Extra 
attention to reading was also provided through programs such as Pegasus, First Steps, 
LEAP, CRISS, and Reading Recovery. They used tutors before, during, and after school; 
on weekends; and in the summer. Computer-assisted reading and mathematics programs 
were used, especially Accelerated Reader. 

Use of technology was frequently evident in these schools. Most had accessible 
computer labs, and many had computers in the classrooms. In one school, the students 
started using computers in kindergarten to learn word processing skills. The teachers 
report that the quality of the student work is greater as students publish their work on 
the computer. The teachers see computers as “great levelers and enablers.” Many 
students reportedly believe that they can compose documents better at the computer. 

Provide Extra Help and Early Intervention: Most of these schools serve 
populations of potentially high risk students, yet failure is not acceptable to these 
schools. They found ways to provide assistance at the first signs of trouble, as well as 
offering tutoring and other sources of extra instruction. After-school tutoring, 

Saturday help sessions, volunteer tutors, and summer academic camps were just several 
of the ways cited by schools to help prevent students from falling too far behind. 

Many of these schools pay special attention to students scoring at Achievement 
Levels 1 and 2 on the EOG Tests, with specific plans to intervene early and often. For 
example, one principal had conferences with parents and students who score at these 
Levels. During the conference, he reviewed the student’s cumulative record and “laid 
out” the student’s past performance in two columns: academic performance and 
attendance. 

Be Willing to Experiment: “If at first you don’t succeed....” Interviewers 

observed that many of these schools were willing to work together to try new 
approaches. If these approaches did not work, they tried something else. However, there 
was no reckless movement from one approach to another. The schools frequently were 
aware of the research in a given area. While they respected the research, they did not 
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follow it rigidly if knowledge of their students suggested something else. One example 
came from a PK-6 school principal about grade retention: “We retain some of our first 
graders if we feel that - for that student - it is the best thing. / know the research! 
However, sometimes we feel that this is the best strategy for a child.” 

Some of the strategies that they are experimenting with either at the individual 
teacher level, grade level, or whole-school level are inclusion, portfolio assessment, 
uniforms, schedule revisions to provide double reading periods and common planning 
time, and teachers staying with the same group of students over a two to three year 
period. They assess as a group whether or not the strategy is working. If, as a group, 
they do not see evidence that it is being successful, they “tweak” it until it is successful, 
or change strategies. 

Do Whatever It Takes: The concern and efforts of staff in these 11 schools 
can best be summed up as “doing whatever it takes.” If one strategy does not work, they 
try something else. Quotes heard in different schools include: “Do whatever it takes...,” 
“Go all the way,” “Don’t back down...,” “Stick to your guns.” “No excuses....” Many 
staff give 100 percent effort, and do not define their work day by the school bell. In fact, 
some teachers expressed concern about their colleagues who do not share these goals, 
who leave when the bell rings at the end of the day, and who are not contributing to the 
growth of staff or of students. 



Summary 

Many of these 11 schools could be characterized as child-oriented, focused on 
individual student achievement, and data driven. They did not necessarily use 
individualized instruction but did show concern for individual student success. There 
was no one instructional strategy that cut across all schools. However, most of them did 
indicate they had moved to more manipulatives in mathematics and used a combination of 
phonics with teaching reading in context and whole language. 

The expectation that all students would learn well was especially common. They 
focused on the NC Standard Course of Study, and used assessment results to evaluate 
student success and instructional strategies. Many schools were willing to try 
approaches based on student needs; but if these strategies did not work, they were 
willing to try again. Concern for students extends into the personal realm, including 
frequent calls to parents, meetings with parents at school, and home visits. Staff work 
with social services or other public agencies to secure needed services for their 
students. Teachers seek continued staff development and new ways to teach. They look to 
the research, and then combine that with their own experience and judgment. They are 
reluctant to ever give up on students. They know that they have a long way to go and do 
not have all the answers; but most teachers clearly enjoy being where they are-trying 
to help students reach excellent levels of learning. 
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Next Steps 



In spite of the positive characteristics and results found in this study, most of 
these schools are still low performing. Where schools did not show some of these 
characteristics, interviewers could not help but wonder what would happen if the 
schools worked more deliberately to incorporate them. Clearly, they have made 
progress but still have a long way to go, as do many schools in the state. 

As the ABCs Accountability Model completes its first year for Grade 3 through 8 
in 1996-97, the Evaluation Section will continue to look for high growth and high 
performing schools, especially higher poverty schools. The need to find models for 
raising student achievement is important to the goal of helping all of our students reach 
high standards of learning. 
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APPENDICES 



Appendix A 





Members of the Improving Schools 
Cross-Functional Team 


Name 


Area Represented 


Delores Brewer 


Evaluation Section, Division of Accountability Services 


Donna Cheek 


Staff Development Section, Division of School 
Improvement 


Carolyn Cobb 


Evaluation Section, Division of Accountability Services 


Jackie Colbert 


Staff Development Section, Division of School 
Improvement 


Martha Downing 


Areas of Exceptionality Section, Exceptional 
Children’s Division 


Bobbye Draughon 


Staff Development Section, Division of School 
Improvement 


Mike Frye 


Language Arts and Social Studies Section, Division of 
Instructional Services 


Jeane Joyner 


Evaluation Section, Division of Accountability Services 


Martha Kincheloe 


Student Information Support Systems Section, Office 
of Information and Technology Services 


Marilyn Palmer 


Staff Development Section, Division of School 
Improvement 


Angela Pope-Lett 


Evaluation Section, Division of Accountability Services 


Ellie Sanford 


Testing Section, Division of Accountability Services 


Susan Temple 


Language Arts and Social Studies Section, Division of 
Instruction'al Services 


Judy White 


Office of Education Reform 


Jan Williamson 


Language Arts and Social Studies Section, Division of 
Instructional Services 


Gongshu Zhang 


Evaluation Section, Division of Accountability Services 
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Appendix B 



Calculating the Index 



Multiply Percent of Students in: Bv: 

Level IV 3 

Level III 2 

Level II 1 



Add the three products together and divide by 3. 



Example: 

Suppose 6% of the students score in Level IV; 48% score in 
Level III; and 34% score in Level II. Then the EOG Index is 
49.3, as shown below. 

6x3 = 18 
48 X 2 = 96 
34 X 1 = 34 



148/3 = 49.3 



Appendix C 

Relationship between INDEX and Percentage of Students in Four Achievement Levels 







CD 

CVJ 




Appendix C (continued) 










1:^ 


CJ 




0) 


XU 


JD 


JD 


B 


B 


E 


p 


p 


p 


c 


c 


c 






4-^ 


c 


c« 


c 


o 


CJ 


O 


T3 


TD 


T3 


B 


3 


3 


CO 


CO 


CO 


3 




a 


o 


p 


o 






fc: 




<n 


O'J 






p I 


o 


o 


o 


> 


> 


> 




o 


3 






.£ 




3 


a 


c 


c 


c 


0) 


0) 


0) 


TD 




T3 


3 


a 


3 


CO 


CO 


oc 








O 


C 


O 


Ui 


Urn 


Ui 


OJ 


o 


O 


jd 


jd 


•£ 


£ 










3 


Z 


Z 

's»-^ 




o 


o 


o 


o 


o 


o 








II 


II 


II 


Cu 


fo 

0- 


(S 

Cu 



* 



0 ^ 

GVJ 



00 

CVf 




Appendix D 



Definition of “Improving” 



Notei “Growth” - Refers to increase in scores for the same cohort of 
over time (e.g., grades 3 to 4; grades 3 to 4 to 5). 110% is based 

on calculations of the “expected growth” based on the ABCs 
formula. This means schools exceeded expected growth by 10%. 

A: 1 10% growth in both Reading and Math from 

grades 3 to 4: 1993 to 94 and/or 1994 to 95. 

B: 1 10% growth in both Reading and Math from 

grades 4 to 5: 1993 to 94 and/or 1994 to 95. 

C: 110% growth in Math only from grades 3 to 4 to 5 

from 1993-95 (e.g., two years in a row) 

D: 1 10% growth in Reading only from grades 3 to 4 to 

5 from 1993-95 (e.g., two years in a row) 

E: 1 10% growth in Reading and Math from grades 6 to 

7: 1993 to 94 and/or 1994 to 95. 

F: 1 10% growth in Reading and Math from grades 7 to 

8: 1993 to 94 and/or 1994 to 95. 

G: 1 10% growth in Math only from grades 6 to 7 to 8 

from 1993-95. 

H: 110% growth in Reading only from grades 6 to 7 to 

8 from 1993-95. 
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Appendix E 



Characteristics of Case Study Schools 
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700 
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School C 
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304 
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678 


58.3 
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School E 


PK-6 


805 


66.8 
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Not Available 


School F 


K-6 


793 


76.0 


98.0 
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School G 


PK-5 


178 


90.3 


94.3 
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School H 


PK-5 


421 
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Status of Meeting Expected/Exemplary Growth for 11 Schools 
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Based on the available data base, these calculations did not include WRITING scores, and did not use pre- & post-test student match or 91 -day rules. 



GROWTH OF INDEX OF ELEVEN SUCCESSFUL SCHOOLS - 1993-1996 
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